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Abstract 

This paper deals with descriptive complexity of picture languages of any dimension by syntactical fragments 
of existential second-order logic. 

• We uniformly generalize to any dimension the characterization by Giammarresi et al. [GRST96] of 
the class of recognizable picture languages in existential monadic second-order logic. 

• We state several logical characterizations of the class of picture languages recognized in linear time 
on nondeterministic cellular automata of any dimension. They are the first machine-independent char- 
acterizations of complexity classes of cellular automata. 

Our characterizations are essentially deduced from normalization results we prove for first-order and exis- 
tential second-order logics over pictures. They are obtained in a general and uniform framework that allows 
to extend them to other "regular" structures. Finally, we describe some hierarchy results that show the 
optimality of our logical characterizations and delineate their limits. 

Keywords: Picture languages, locality, recognizability, linear time, cellular automata, logical 
characterizations, existential second-order logic. 



Introduction 

One goal of descriptive complexity is to establish logical characterizations of natural classes of problems 
in finite model theory. Many results in this area involve second-order logic (SO) and its restrictions, monadic 
second-order logic (MSO) and existential second-order logic (ESO). Indeed, there are two lines of research 
that roughly correspond to either of these restrictions: 

(a) The formal language current. It starts from the pioneering result by Biichi, Elgot and Trahtenbrot [Biic60], 
which states that the class of regular languages equals the class of MSO-definable languages; in short, 
REG = MSO. This line of research aims at characterizing in logic the natural classes of algebraically de- 
fined languages (sets of words) or sets of structures (trees, graphs, etc.) defined by finite state recognizability 
or local properties such as tilings. 
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(b) The computational complexity current. It originates from another famous result, Fagin's Theorem [Fag74], 
which characterizes the class NP as the class of problems definable in ESO. 

For many years, both directions of research have produced plenty of results: see e.g. [EF95, Lib04] for 
descriptive complexity of formal languages and [EF95, Imm99, GKL + 07, Lib04] for the one of complexity 
classes. However, and this may be surprising, only few connections are known between those two areas of 
descriptive complexity. Of course, an explanation is that formal language theory has its own purposes that 
have little to do with complexity theory. In our opinion, the main reason is that while MSO logic exactly 
fits the fundamental notion of recognizability, as exemplified in the work of Courcelle [CE12], this logic 
seems transversal to computational complexity. We argue this is due to the intrinsic locality that MSO logic 
inherits from first-order logic [Han65, Gai82]. Typically, whereas MSO, or even existential MSO (EMSO), 
expresses some NP-complete graph problems such as 3-colourability, it cannot express some other ones such 
as Hamiltonicity (see [Tur84, dR87, Lib04]) or even some tractable graph properties, such as the existence 
of a perfect matching. In contrast, the situation is very clear on trees as on words: MSO only captures the 
class of "easiest" problems; an extension of Biichi's Theorem [TW68] states that a tree language is MSO 
definable iff it is recognizable by a finite tree automaton 

Thus, items (a) and (b) above seem quite separate for problems on words, trees or graphs, in one case (for 
words and trees) because MSO only expresses easy problems (regular languages), in the other case because 
MSO and EMSO do not correspond to any complexity class over graphs. 

What about picture languages, that is sets of li-pictures, i.e., li-dimensional words (or coloured grids)? 
First, notice the following results: 

1. In a series of papers culminating in [GRST96], Giammarresi and al. have proved that a 2-picture 
language is recognizable, i.e. is the projection of a local 2-picture language, iff it is definable in 
EMSO. In short: REC 2 = EMSO. 

2. In fact, the class REC 2 contains some NP-complete problems. More generally, one observes that 
for each dimension d > 1, REC c/ can be defined as the class of af-picture languages recognized by 
nondeterministic cZ-dimensional cellular automata in constant time 1 . 

In some sense, the present paper is an attempt to bridge the gap between the formal language current of 
descriptive complexity involving MSO and the computational complexity current that involves ESO. This 
paper originates from two questions about word/picture languages: 

1. How can we generalize the proof of the above-mentioned theorem of Giammarresi and al. to any 
dimension? That is, can we establish the equality REC rf = EMSO for any d > 1 ? 

2. Can we obtain logical characterizations of time complexity classes of cellular automata 2 ? 

The paper addresses both questions; it also compares, in a common framework, the point of view of formal 
language theory with that of computational complexity. A af-picture language over an alphabet S is a set 



That means: for such a picture language L, there is some constant integer c such that each computation stops at instant c, and p e L 
iff it has at least one computation whose final configuration is accepting in the following sense: all the cells are in an accepting state. 

2 This originates from a question that J. Mazoyer asked us in 2000 (personal communication): give a logical characterization of the 
linear time complexity class of nondeterministic cellular automata. 
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of of-pictures p : [l,ri\ d — > S, i.e., cZ-dimensional 2-words 3 . There are two natural manners to represent a 
(f-picture p as a first-order structure: 

• as a pixel structure: on the pixel domain [l,n] c/ ; 

• as a coordinate structure: on the coordinate domain [1,«]. 

Significantly, these two representations respectively correspond to the two above-mentioned points of 
view as shown by our results. 

Our results: 

We establish two kinds of logical characterizations of of-picture languages, for all dimensions d > 1 : 

1. On pixel structures: REC c/ = ESO(arity 1) = ESO(var 1) = ESO(V 1 ,arity 1). That means a af-picture 
language is recognizable iff it is definable in monadic ESO (resp. in ESO with 1 first-order variable, 
or in monadic ESO with 1 universally quantified first-order variable). 

2. On coordinate structures: NLIN^ a = ESO(varc/+ 1) = ES O (V*" 1 " 1 , arity c/ + 1); that means a cZ-picture 
language is recognized by a nondeterministic af-dimensional cellular automaton in linear time iff it is 
definable in ESO with d + 1 distinct first-order variables (resp. ESO with second-order variables of 
arity at most d+ 1 and a prenex first-order part of prefix V rf+1 ). 

Both items (1) and (2) are easy consequences of normalization results of, respectively, first-order and 
ESO logics we prove over picture languages. In particular, the "normalization" equality ESO(arity 1) = 
ESO(V\ arity 1) is a consequence of the fact that on pixel structures (and more generally, on structures that 
consist of bijective functions and unary relations) any first-order formula is equivalent to a boolean combi- 
nation of "cardinality" formulas of the form: "there exists k distinct elements x such that ijf(x)", where tf/ is a 
quantifier-free formula with only one variable. The "normalization" equality explicitly expresses the "local" 
feature of MSO on pictures and can be generalized to other "regular" structures. 

Thirdly, in contrast with (1) and (2), we establish several strict hierarchy results for any fixed d > 2 and 
for cZ-picture languages represented by coordinate structures: in particular, we prove 

ESO(var<i- 1) c REC 1 c ESO(vani) = NLIN^ a 

and ESO(varoT) c ESO(arity d). Here, the fact that no natural restriction of ESO (in particular, ESO(varcf)) 
exactly captures the class REC rf for the coordinate representation of pictures seems to us the symptom of the 
large expressiveness of this logic: significantly, the strict inclusion ESO(var d) c ESO(arity d) for of-pictures 
in the coordinate representation, if d > 2, strongly contrasts with the equality ESO(arity 1) = ESO(var 1) that 
holds in the pixel representation. 

In this document, we sketch many proofs and omit other ones, in particular, the proofs of hierarchy 
theorems, in Section 4. However, the very technical proof of the normalization of the logic ESO(Vc/, arity d) 
on coordinate encodings of (d- l)-pictures (for d > 2) is completely described in the appendix. 



3 More generally, the domain of a ^-picture is of the "rectangular" form [l,»i] X ... X [l,nd]. For simplicity and uniformity of 
presentation, we have chosen to present the results of this paper in the particular case of "square" pictures of domain [n\ d ■ Fortunately, 
our results also hold with the same proofs for general domains [l,«i]X...X [l,nj]. 
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1. Preliminaries 

In the definitions below and all along the paper, we denote by E, F some finite alphabets and by d a 
positive integer. For any positive integer n, we set [n] := {1, . . . ,«}. We are interested in sets of pictures of 
any fixed dimension d. 

Definition 1.1. A cZ-dimensional picture or <f -picture on £ is a function p : [n] d — > S where n is a positive 
integer. The set dom(p)— [n] d is called the domain of picture p and its elements are called points, pixels or 
cells of the picture. A set of d-pictures on S is called a af-dimensional language, or li-language, on S. 

Notice that 1 -pictures on £ are nothing but nonempty words on S. 

1.1. Pictures as model theoretic structures 

Along the paper, we will often describe li-languages as sets of models of logical formulas. To allow this 
point of view, we must settle on an encoding of li-pictures as model theoretic structures. 

For logical aspects of this paper, we refer to the usual definitions and notations in logic and finite model 
theory (see [EF95] or [Lib04], for instance). A signature (or vocabulary) cr is a finite set of relation and 
function symbols each of which has a fixed arity. A (finite) structure S of vocabulary cr, or cr-structure, 
consists of a finite domain D of cardinality n > 1, and, for any symbol s e cr, an interpretation of s over 
D, often denoted by s for simplicity. The tuple of the interpretations of the cr-symbols over D is called the 
interpretation of cr over D and, when no confusion results, it is also denoted cr. The cardinality of a structure 
is the cardinality of its domain. For any signature cr, we denote by struc(ct) the class of (finite) cr-structures. 
We write models(O) the set of cr-structures which satisfy some fixed formula <D. We will often deal with 
tuples of objects. We denote them by bold letters. 

There are two natural manners to represent a picture by some logical structure: on the domain of its 
pixels, or on the domain of its coordinates. This gives rise to the following definitions: 

Definition 1.2. Given p : [n] d — > S, we denote by pixel (p) the structure 

pixef'ip) = ([nf, (Q s ) s ei.Jsucci) ie[d] ,(min i )i €[dh (maxi)i € [d]). 

Here: 

• succj is the (cyclic) successor function according to the j dimension of[n] d , that is: for each a — 
(ai)ie[d] 6 [n] d , succj(a) = a^ where a. , the i ,h component of a^\ equals a,-, the i' h component a,- of 
a, except the j th one which equals the cyclic successor of the j th component aj of a. More formally: 

— a^ — aj + 1 if aj < n, and aP — 1 otherwise; 

— a. — aifor each i + j. 

• the mini 's, max, 's and Q s 's are the following unary ( monadic) relations: 

— min/ = {a e [ri] '■ a i - 1)/ 

— maxj — {a € [n] d : a,- = «,}; 

— Q s = {a € [n] d : p(a) = s}. 
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Definition 1.3. Given p : [n] d — > S, we denote by coord'' 1 
COOrd'ip) = {[n],(Q s ) se ^,<,succ,min,max). 



(p) the structure 



(1) 



Here. 



• £ac/i ;s a d-ary relation symbol interpreted as the set of cells of p labelled by an s. In other words: 



• <, min, max are predefined relation symbols of respective arities 2, 1, 1, that are interpreted, respec- 
tively, as the sets {(i,j) :l<i< j<n}, {1} and {«}. 

• succ is a unary function symbol interpreted as the cyclic successor. (That is: succ(i) = i+ 1 for i < n 
and succ(n) — I.) 

For a of-language L, we set pixef'iL) = {p\xe\ d (p) : peL] and coord'(L) = {coord rf (p) : p e L). 

1.2. Logics under consideration 

Let us now come to the logics involved in the paper. All formulas considered hereafter belong to rela- 
tional Existential Second-Order logic. Given a signature <x, indifferently made of relational and functional 
symbols, a relational existential second-order formula of signature <x has the shape O = 3Rtf(cr, R), where 
R = (R\ , . . .,Rk) is a tuple of relational symbols and <p is a first-order formula of signature crU {R}. We denote 
by ESO ^ the class thus defined. We will often omit to mention <x for considerations on these logics that do 
not depend on the signature. Hence, ESO stands for the class of all formulas belonging to ESO ^ for some 
o~ . 

We will pay great attention to several variants of ESO. In particular, we will distinguish formulas of type 
Q> = 3R<p(<T, R) according to: 

- the number of distinct first-order variables involved in <p, 

- the arity of the second-order symbols R eR, and 

- the quantifier prefix of some prenex form of (p. 

With the logic ESO £r (V f/ , arity £), we control these three parameters: it is made of formulas of which 
first-order part is prenex with a universal quantifier prefix of length d, and where existentially quantified 
relation symbols are of arity at most t. In other words, ESO^fV , arity €) collects formulas of shape: 



where 6 is quantifier free, x is a of-tuple of first-order variables, and R is a tuple of relation symbols of arity 
smaller than £. Relaxing some constraints of the above definition, we set: 



Finally, we write ESO' r (varii) for the class of formulas that involve at most d first-order variables, thus 
focusing on the sole number of distinct first-order variables (possibly quantified several times). 



Q s = {a e [n] d : p(a) = s). 



3RVx6»(<x,R,x) 




d>Q 




5 



2. A logical characterization of REC 

In order to define a notion of locality based on sub-pictures we need to mark the border of each picture. 

Definition 2.1. By Y$ we denote the alphabet Y U {ft} where ft is a special symbol not in Y. Let p be any d- 
picture of domain [n] d on Y. The bordered of-picture of p, denoted by p*, is the d-picture p" : [0,n + l] d — > Y$ 
defined by 

^ a 1 ft otherwise. 

Here, "otherwise" means that a is on the border of that is, some component ai of a is Oorn + l. 

Let us now define our notion of local picture language. It is based on some sets of allowed patterns 
(called tiles) of the bordered pictures. 

Definition 2.2. 1. Given a d-picture p and an integer j e [d], two cells a — (a,)ie|i/] and b — (Z>j)ie[d] of p 
are y'-adjacent if they have the same coordinates, except the j th one for which \aj-bj\ = 1. 

2. A tile for a d-language LonY is a couple in (F") 2 . 

3. A picture p is y'-tiled by a set of tiles A C (r") 2 if for any two j-adjacent points a,b € dom(p^): 
( P Ha),pHb))eA. 

4. Given d sets of tiles A\ , . . . , c (r") 2 , a d-picture p is tiled by (Aj ,. . . ,A^) ;/ p is j -tiled by A; for 
each j 6 [d]. 

5. We denote by L(Ai,. . . ,A f /) the set of d-pictures on Y that are tiled by (Ai,. . . ,A,j). 

6. A d-language L onY is local if there exist Ai,..., Ad £ (T") 2 such that L — L(A\,. . . ,A ( /). We then say 
that L is (Ai , . . . , A^)-local, or (A\ , ... , A^)-tiled. 

Definition 2.3. A d-language L on 1. is recognizable it is the projection of a local d-language over an 
alphabet Y. It means there exist a surjective function n : Y — > 2 and a local d-language L[ oc on Y such that 
L = {7Top; peLioc). 

Because of Definition 2.2, it also means there exist a surjective function n : Y — > E and d subsets A\,...,A c i 
of(Y$) 2 such that 

L = {nop; peL(A 1 ,...,A d )}. 
We write REC' 1 for the class of recognizable d-languages. 

Remark 2.4. Our notion of locality is weaker than the one given by Giammarresi and al. [GRST96]. But 
this doesn 't affect the meaning of recognizability, which coincides with that used in [GRST96]. This confirms 
the robustness of this latter notion. 

A characterization of recognizable languages of dimension 2 by existential monadic second-order logic 
was proved by by Giammarresi et al. [GRST96]. They established: 

Theorem 2.5 ([GRST96]). For any 2-dimensional language L, Le REC 2 o pixel 2 (L) e ESO(arity 1). 

In this section, we come back to this result. We simplify its proof, refine the logic it involves, and generalize 
its scope to any dimension. 
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Theorem 2.6. For any d > and any d-language L, the following assertions are equivalent: 

1. LeREC 1 ; 

2. p/xe/^eESOO/ 1 , arity 1); 

3. pkef'(L) e ESO(arity 1). 

Proposition 2.7 states the equivalence 1 o 2. In Proposition 3.9, we establish the normalization ESO(arity 1) = 
ESO(V', arity 1) on pixel structures, from which the equivalence 2 <=> 3 immediately follows. 

Proposition 2.7. For any d>0 and any d-language L on E: L e REC rf <=> p/xe/ rf (L) € ESOCV 1 , arity 1). 

Sketch of Proof. => | A picture belongs to L if there exists a tiling of its domain whose projection 
coincides with its content. In the logic involved in the proposition, the "arity 1" corresponds to formulating 
the existence of the tiling, while the V 1 is the syntactic resource needed to express that the tiling behaves as 
expected. Let us detail these considerations. 

By Definition 2.2, there exist an alphabet F (which can be assumed disjoint from £), a surjective function 
7T : T -» E and d subsets Ai,..., Ad C (r") 2 such that L is the set {nop' : p' e L(Ai, . . . , Ad)}. 

The belonging of a picture p' : [n] d — > F toL(Ai, . . . , A c /) is easily expressed on p\xe\ d (p') - {[n] d ,(Q s )ser, ■ ■ ■) 
with a first-order formula which asserts, for each dimension i e [d], that for any pixel i of p', the couple 
(x,succ,(x)) can be tiled with some element of A, . Because it deals with each cell x separately, this formula 
has the form Vx v F(x,(g s ) se r), where *P is quantifier-free. 

Now, a picture p : [n] d — > £ belongs to L iff it results from a 7r-renaming of a picture p' € L(Ai , . . . , A^). It 
means there exists a T-labeling of p (that is, a tuple (Q s ) S €r of subsets of [n] d ) corresponding to a picture of 
L(Ai,...,A ( /) (i.e. fulfilling Vjc v F(x,(2 s ) seE r)) and from which the actual E-labeling of p (that is, the subsets 
(Qs)s€i.) is obtained via n (easily expressed by a formula of the form Vx*P'(x,(g.s)sei>((2j)ser))- 

Finally, the formula (3Q s )s€rVx : *P A conveys the desired property and fits the required form. 



<= | In order to prove the converse implication, it is convenient to first normalize the sentences of 

ESO(V 1 , arity 1). This is the role of the technical result below, which asserts that on pixel encodings, each 
such sentence can be rewritten in a very local form where the first-order part alludes only pairs of adjacent 
pixels of the bordered picture. We state it without proof: 

Fact 2.8. On pixel structures, any ip € ESCKV 1 , arity 1) is equivalent to a sentence of the form: 





mini(x) 
maxi(x) 
-imaxi(x) 



— > rrij(x) A 

-> Mi(x) A 




(2) 



Here, U is a list of monadic relation variables and /T)„ Mj, are quantifier-free formulas such that 



• atoms of mj and Mi have all the form Q(x); 



• atoms of^i have all the form Q(x) or Q(succi(x)), 



where, in both cases, Q e {(Q s )sel,U}. 
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Now, consider L such that p\xe\ d (L) e ESO(V 1 ,arity 1). Fact 2.8 ensures that pixel''(L) is characterized 
by a sentence of the form (2) above. We have to prove that L is the projection of some local cZ-language L\ oc 
on some alphabet T, that is a (Ai,. . . , A f /)-tiled language for some Ai,. .., A ( / c T 2 . Let U\,...,Uk denote 
the list of (distinct) elements of the set {(Q s ) s€ z,U} of unary relation symbols of ip so that the first ones 
U\,...,U m are the Q s 's (here, min, and max, symbols are excluded). The trick is to put each subformula 
m,(.x), M,(x) and of tp into its complete disjunctive normal form with respect to U\,...,Uk- Typically, 
each subformula *P,-(jt) whose atoms are of the form U,(x) or i/,(succ,(x)), for some j e [k], is transformed 
into the following "complete disjunctive normal form": 



\/ /\ ejUjix) A /\ e'jU j(succ,(x)) 
(e,e')6AiUeM je[k] 



(3) 



Here, the following conventions are adopted: 

• e = (ex , . . . , efc) 6 {0, 1}* and similarly for e' ; 

• for any atom a and any bit ej e {0, 1), eja denotes the literal a if ej ■ - 1, the literal -kt otherwise. 

For e G {0, 1}^, we denote by & e (x) the "complete conjunction" Aj€[k] e jUj(x). Intuitively, ® e (x) is a complete 
description of x and the set F = U;e[m]{0' -1 10 m-1 } x {0, \) k ~ m is the set of possible colors (remember that the 
(2/s that are the Ufs for j e [m] form a partition of the domain). The complete disjunctive normal form (3) 
of *Fi(x) can be written into the suggestive form 



\/ (©,(*) A e <(succ,(x))). 



(e.e')eAi 



If each subformula m,(jic) and M,(x) of ip is similarly put into complete disjunctive normal form, that is 
V(tt,e)eA, :®fW and V(e,(|)€A, ■ respectively (there is no ambiguity in our implicit definition of the A,'s, 

since ft £ F), then the above sentence (2) equivalent to <p becomes the following equivalent sentence: 



iff = 3UVx/\ 



mmi(x) -> Y e (x) 

0U)eAi 
max,(x) Y 

-imaxj(^) — > \y (®eW A0 £ '(succ/(jc))) 



Finally, let Li oc denote the (i-language over F defined by the first-order sentence y>] oc obtained by replacing 
each e by the new unary relation symbol Q 6 in the first-order part of <p'. In other words, pixel''(Li oc ) is 
defined by the following first-order sentence: 



Woe = VxAfewr 



min,(x) -> Y 
max,(x) Y 

(e.tt)GA,- 

nmax,(x) -> Y (GeWAg e '(succ,(x))) 

(e,e')eAi 



Hence, L\ oc = L(Ai A c i). That is, L\ oc is indeed local and the corresponding sets of tiles are the A,'s of the 
previous formula. It is now easy to see that our initial cZ-language L is the projection of the local language 
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^loc by the projection n : F — > 2 defined as follows: 7r(e) = 5 iff e, ■ — 1 for i e [m] and {/,• is Q s . This completes 
the proof. □ 



Proposition 2.9. ESO(arity 1) c ESO(V , arity 1) on pixe/ structures, for any d>0. 

Sketch of Proof. In a pixel structure, each function symbol succ, is interpreted as a cyclic successor, 
that is a bijective function. So, a pixel structure is a bijective structure, that is a first-order unary structure 
whose (unary) functions / are bijective and that explicitly includes all their inverse bijections f~ l . It has 
been proved in [DG06] that any first-order formula on a bijective structure can be rewritten as a so-called 
cardinality formula, that is as a boolean combination of sentences of the form if/- k - 3- k x i//(x) (for k > 1) 
where ifr(x) is a quantifier-free formula with the single variable x and where the quantifier 3- k x means "there 
exist at least k elements x". Therefore, it is easily seen that proving the proposition amounts to show that 
each sentence of the form tp~ k or -^\f/- k can be translated in ESO(V', arity 1) on pixel structures. 

This is done as follows: for a given sentence 3- k x ifr(x), we introduce new unary relations U =0 , U =1 ,...,U =k ~ 
and U~ k , with the intended meaning: 

A pixel a e [n] d belongs to U = i ( resp. U ) iff there are exactly j ( resp. at least k) pixels be [n] d 
lexicographically smaller than or equal to a such that pixer(p) \= i//(b). 

Then we have to compel these relation symbols to fit their expected interpretations, by means of a first-order 
formula with a single universally quantified variable. First, we demand the relations to be pairwise disjoint: 



Then, we temporarily denote by <i ex the lexicographic order on [n] d inherited from the natural order 
on [n], and by succi ex , mini ex , maxi ex its associated successor function and unary relations corresponding 
to extremal elements. Then the sets described above can be defined inductively by the conjunction of the 
following six formulas: 



(1) 




i<j<k 



i<k 



(2) (min lex (*) A ^(x)) ^ U=°(x) 



(3) (mini ex (*)A^(*))->t/ =1 



(a) 




i<k 



(5) f\ (-imaxiaC*) A U '(x)Atff(succi ex (x)))-^U ,+1 (succ lex (x)) 



i<Jfe-l 



(6) (-imaxie X (*) A U k l (x) A i/c(succi ex (x))) -> U- k (succ\ ex (x)) 



(7) (imaxiaOc) A U**(xj) -» U^ k (succ lex (x))- 
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Hence, under the hypothesis (1)A...A(7), the sentences \f/- k and -^\f/- k are equivalent, respectively, to 
Vx(maxiexO) — > U- k {x)) and Vx(maxi ex (x) — > -iU- k (xJ). 

To complete the proof, it remains to get rid of the symbols succi ex , mini ex and maxi ex that are not allowed 
in our language. It is done by referring to these symbols implicitly rather than explicitly. For instance, since 
succi ex (x) = succ,-sucCj + i . . . succ c /(x), for each non maximal x e [n] d , i.e., distinct from («,...,«), and for 
the smallest i e [d] such that /\ J>( max ; (x), each formula <p involving succi ex (x) actually corresponds to the 
conjunction: 

y\ (-imax/(jf) A /\ imxjix)) w , 

i£[d]\ i<j<d 

where <fi is obtained from tp by the substitution succ] ex (x) -w* succ,succ, + i ...succrf(x). Similar arguments 
allow to get rid of miniex and max] ex . □ 

Remark 2.10. In this proof, two crucial features of a structure of type pixefi{p) are involved: 

• its "bijective" nature, that allows to rewrite first-order formulas as cardinality formulas with a single 
first-order variable; 

• the "regularity" of its predefined arithmetics (the functions succi defined on each dimension), that 
endows pixe^ip) with a grid structure: it enables us to implicitly define a linear order of the whole 
domain dom(p) by means of first-order formulas with a single variable, which in turn allows to express 
cardinality formulas by "cumulative" arguments, via the sets U =l . 

Proposition 3.9 straightforwardly generalizes to all structures - and there are a lot - that fulfill these two 
properties. 

To conclude this section, let us mention that we could rather easily derive from Theorem 2.6 the following 
additional characterization of REC'': 

Corollary 2.11. For any d > and any d-language L, the following assertions are equivalent: 

1. LsREC d ; 

2. pixef'iL) e ESO(var 1). 



3. A logical characterization of NLIN ca 

Besides the notion of recognizable picture language, the main concept studied in this paper is the linear 
time complexity on nondeterministic cellular automaton of any dimension. 

Definition 3.1. A one-way ^-dimensional cellular automaton fii-automaton, for short) over an alphabet 2 is 
a tuple J{ — (L,r,6,F), where 

• the finite alphabet T called the set of states of J{ includes the input alphabet S and the set F of 
accepting states: X,Fc T; 

• 5 is the (nondeterministic) transition function of 3K: 6 : T x (T^) d — > P(T). 
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Definition 3.2. Let J[ = (Z,Y,6,F) be a d-automaton and p,p' : [n] d — > T be two d-pictures on Y. We say 
that p' is a successor of p for J[, denoted by p' € M(p), if for each a e [n] d , 

p'(a)e6(p(a),pHa m ),...,pHa (d) )), 

where p$ denotes the bordered picture of p and, for i € [d], a^ denotes the point having the same coordinates 
as point a except the i th one which is a,- + 1. So, the value of p' at point a e [n] d only depends on the set of 
points a,cP~\ . . . of p. This set ofd+1 points is called the neighborhood of a. 
The set of /''-successors of p for J[, denoted 3V(p), is defined inductively: 

3l (p) = {p}andforj>0:3l j+1 (p)= (J Jl(p'). 

Definition 3.3. A computation of a d-automaton J{ — (T,,T,6,F) on an input d-picture p onl. is a sequence 
Pi, Pi, Pi, ■ ■ ■ of d-pictures on T such that pi — p and pt+i € Jiipi) for each i. The picture pi, i > 1, is 
called the i tfl configuration of the computation. A computation is accepting if it is finite — it has the form 

Pl,P2, ■ ■ - ,Pkf or some k — and the cell of minimal coordinates, 1 = (1 1), of its last configuration is in 

an accepting state: pk(l d ) € F. 

Remark 3.4. Note that the space used by a d-automaton is exactly the space (set of cells) occupied by its 
input d-picture. 

Definition 3.5. Let tft — (I.,T,6,F) be a d-automaton and let T : N — » N be a function such that T(n) > n. A 
d-picture p : [n] d — > 2 is accepted by J{ in time T if admits an accepting computation of length T(n) on 
p. That means, there exists a computation p — p\,p2, ■ ■ - ,PT(n) ofM on p such that PT(n)(^ d ) e F. 

A d-language L on~L is accepted, or recognized, by J{ in time T if it is the set of d-pictures accepted by 

in time T. That is 

L = {p:3p' € J[ T{n) - l (p) such that p'(l d ) e F where dom(p) = [n] d ). 

If T(n) — cn + c' , for some integers c, c' , then L is said to be recognized in linear time and we write L e 
NLI<. 

The time bound T(n) > n of the above definition is necessary and sufficient to allow the information of 
any pixel of p to be communicated to the pixel of minimal coordinates, \ d . 

Remark 3.6. The nondeterministic linear time class NLIN^' a is very robust, i.e. is not modified by many 
changes in the definition of the automaton or in its time bound. In particular, the constants c,c' defining 
the bound T(n) — cn + c' can be fixed arbitrarily, provided T(n) > n. Typically, the class NLIN^, does not 
change if we take the minimal time T(n) — n + l, called real time. 

As we have stated several logical characterizations of the class REC d of recognizable picture languages 
(Theorem 2.6), we now give two characterizations of the class NLIN^ in existential second-order logic. 

Theorem 3.7. For any d > and any d-language L, the following assertions are equivalent: 

1. LeNLIN^; 

2. coord'(L) e ESO(y +1 ,arity d+ 1); 
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3. coord' (L) e ESO(var + 1). 

This theorem is a straightforward consequence of Proposition 3.8 and 3.9 below. The former states 
the equivalence (1) <=> (2); with the latter, we establish the normalization ESO(varcf) c ESO(V c/ , arity d) on 
coordinate structures, which yields equivalence (2) <=> (3). 

Proposition 3.8. For any d > and any d-language L, 

L e NLlN d . a o coord' (L) e ESO(V d+1 , arity d+ 1). 

Sketch of Proof. [5] Let L e NLIN^. By Remark 3.6, let us assume without loss of generality that L is 
recognized by a li-automaton M = (E,T,i5,F) in time n+ 1. The sentence in ESCKV^ 1 , arity of + 1) that we 
construct is of the form 3(R s ) s€ r Vx0"(x), where i/f(x) is a quantifier-free formula such that: 

• if/ uses a list of exactly d+ 1 first-order variables x = (jci , , jc^, JCrf+i). Intuitively, the d first ones 

represent the coordinates of any point in dom(p) = [n] d and the variable Xd+i represents any of the first 
n instants t e [n] of the computation (the last instant n + 1 is not explicitly represented); 

• if/ uses, for each state s e F, a relation symbol R s of arity d + 1 . Intuitively, R s (a\,.,., a^, t) holds, for 
any a = (a\,.. . ,a</) e [n] d and any t e [«], iff the state of cell a at instant t is s. 

i/r(x) is the conjunction if/(x) = init(x) A step(x) A end(x) of three formulas whose intuitive meaning is the 
following. 

• Vx init(x) describes the first configuration of M, i.e. at initial instant 1; 

• Vx step(x) describes the computation of J[ between the instants t and t + 1, for t e [n — 1], i.e. describes 
the (t+ \) th configuration from the t th one; 

• Vx end(x) expresses that the n th configuration of J[ leads to a (last) (n + \) th configuration which is 
accepting, i.e. with an accepting state in cell l d . 

Let us give explicitly the main formula step. The main technical difficulty comes from the description 
of the transition function 5 of J[ for any cell in the border of the picture. In order to describe uniformly 5 
we introduce the notion of neighborhood. A neighborhood for !A is any function v : [0,d] — > P such that 
v(0) € T. Let neighb(Jl) denote the set of neighborhoods v = (v(0),v(l), . . .,v(d)) of J[. So, 5 is a function 
of prototype neighb(yt) -> P(T). 

Clearly, the universal closure Vxstep(x) of the following formula step(x) — that uses the "neighborhood" 
notation - correctly describes the computation between the instants t and t + 1, for t e [n— 1]: 



-» 0*,(x^). 

S€(S(y) 



Here, (J) denotes the exclusive disjunction and x^ ! ' is the (d+ l)-tuple x where x, is replaced by succ(xi). 
Hence we have proved that for any li-picture p on 2, coord' < (/?) satisfies the ESO(V rf+1 , arity d+ l)-sentence 
3(R s ) s€ r s /x ifr(x) if, and only if, ^M n (p) contains an accepting configuration, i.e. J{ accepts p in time n + 1, 
or, by definition, p e L. Hence L e ESO(V rf+1 , arity d+ 1), as required. 



imax(x,/ + i)A/? v(0 )(x) 



step(x) = A 

veneighb(J?I) 



f\ (-max(* i )A.R v(0 (x©)) A 

ie[d]:v(0er 

y^y max(xi) 

ie\dV.v(i)=H 
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<= I Assume coord f/ (L) e ESO(V c/+1 , arity d+ 1), i.e. there is some sentence O e ESO(V c/+ parity d + 1) 
such that peLo coord rf (/?) |= <£. We want to prove L e NLINJ.' a , i.e. L is recognized by some li-automaton 
in linear time. Let us give the main idea of the proof for the simplest case d — 1 and a formula <D € 
ESO(V 2 , arity 2) of the form 

where R is a binary relation symbol and iff is a quantifier-free formula where the only atoms in which R 
occurs, called R atoms, are of the following forms (1-4): 

(l)R(x,y); (2) R(succ(x),y); (3) R(x, succ(y)); (4)R(y,x). 

First, notice that if the only atoms where R occurs are of the forms (1-3), i.e. the variables x,y only appear 
in this unique order in the arguments of R, then formula <t> has a local behaviour: points (x,y), (succ(x),y) 
and (x,succ{y)) are neighbours, i.e. adjacent each other. This allows to construct a 1-automaton (nondeter- 
ministic cellular automaton of dimension 1) J[ that "mimics" <t>. Roughly, J{ "guesses" successively "rows" 
R(i, . . .), for i = 1, 2, . . . n, of R, and in the same time, it "checks locally" the coherence of each "instantiation" 
iff(i, j): more precisely, at "instant" i, the state of each cell j, 1 < j < n, of "contains" both values R(i, j) and 
R(i+ So, in case 7?-atoms are of the forms (1-3), the language L is recognized by such a 1-automaton 
J{ in linear time as claimed. 

Now, let us consider the "general" case where the formula includes all the forms (1-4). Of course, the 
"pixel" (4) R(y,x) is not adjacent to "pixels" (1-3) but it is their symmetric (more precisely, it is the symmetric 
of R(x,y)) with respect to the "diagonal" x — y. The intuitive idea is "to cut" or "to fold" the "picture" R along 
this diagonal: R is replaced by its two "half pictures" denoted R\ and R2, that are "superposed" in the "half 
square" x < y above the diagonal. More precisely, Ri and R2 are binary relations whose intuitive meaning is 
the following: for points (x,y) such that x < y, one has the equivalence Ri(x,y) <=> R(x,y) and the equivalence 
Ri(x,y) <=> R(y,x). By this transformation, each pixel R2(x,y) that represents the original pixel R(y,x) lies at 
the same point (x,y) as pixel R\{x,y) that represents pixel R(x,y), for x < y (the case y < x is similar). This 
solves the problem of vicinity. 

More precisely, the sentence (D = 3RVxVyif/(x,y) is normalized as follows. Let coherent(x,y) denote the 
formula 

x = y -> (Ri(x,y) <-> R 2 (x,y)) 

whose universal closure ensures the coherence of Ri and R2 on the common part of R they both represent, 
that is the diagonal x-y. Using R\ and R2, it is not difficult to construct a formula 



iff'(x,y) ■■ 



coherent(x,y) A ^ 

x<y -*iff<(x,y) A 

x = y — > \f/={x,y) A 
I x >y -> if/>(x,y) 



such that the sentence <t>' = 3Ri3R2^xiyiff'(x,y) in ESO(V 2 , arity 2) is equivalent to O. Let us describe and 
justify its precise form and meaning. 

The formulas i/r<(x,y), tf/=(x,y) and i^>(x,y) are obtained from formula if/{x,y) by substitution of R atoms 
by Ri or R2 atoms according to the cases described in Table 1. It is easy to check that each replacement is 
correct according to its case. For instance, it is justified to replace each atom of the form R(x, succ(y)) in \p 
by /?2(succ(y), x) when x > y (in order to obtain the formula i/f>(x,y)) because when x > y, then succ(y) < x 
and hence the equivalence R(x, succ(y)) <-> /?2(succ(y),x) holds, by definition of R2- 

Notice that the variables x,y always occur in this order in each Ri or R2 atom of the formulas ijj < and 
i// = (see Table 1). At the opposite, they always occur in the reverse order y,x in the formula i^ > (x,y). This 
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Table 1: Replacement ofS-atoms by R[ - orR2-atoms 



case 


formula 


R(x,y) 


R(succ(x),y) 


R(x, succ(y)) 


R(y,x) 


x < y 


</f<(x,y) 


Ri(x,y) 


Ri(succ(x),y) 


Ri(x,succ(y)) 


Ri(x,y) 


x — y 


</f=(x,y) 


Ri(x,y) 


R2(x, succ(y)) 


/?i(x,succ(y)) 


Ri(x,y) 


x>y 


>/f>(x,y) 


Ri(y,x) 


R2(y, succ(x)) 


/?2(succ(y),x) 


Ri(y,x) 



is not a problem because, by symmetry, the roles of x and y can be exchanged and the universal closure 
VxVy(x > y — > i/' > (x,y)) is trivially equivalent to VxVy(y > x — > t/r>(y,x)). So, the above sentence 3>' - and 
hence, the original sentence O - is equivalent to the sentence denoted €>" obtained by replacing in <t>' the 
subformula x > y — > </i>(x,)0 by y > x — > (fr>(y,x). By construction, relation symbols R\,R2 only occur in <t> 
in atoms of the three required "sorted" forms: Ri{x,y), /?,(succ(x),y) or Ri{x, succ(y)). Finally, to be precise, 
there remain two difficulties so that a 1 -automaton can simulate the "sorted" sentence d>" in linear time, by 
the informal algorithm described above: 

• the presence of equalities and inequalities in the sentence; 

• the forms of the atoms involving input relation symbols. 

It is easy to get rid of equalities and inequalities by introducing new binary relation symbols defined and used 
in a "sorted" manner too (see (1-3)). Concerning the second point, we can assume, without loss of generality, 
that the only atoms involving the input relation symbols (Q s ) s€ i, are of the two forms Q s (x) or Q s (y). As we 
do for equalities and inequalities, we can get rid of atoms of the form Q s (y) by introducing new binary ESO 
relation symbols : intuitively, they convey each bit Q s (a) at each point of coordinates (a, . . .) or (. . . ,a); those 
new binary relations are also defined and used in a "sorted" manner. The fact that all the atoms involving the 
input are of the form Q s (x) allows to consider this input in the initial configuration of the computation of the 
1 -automaton but in no later configuration as required. So, the sketch of proof is complete for the case d=l. 

For the general case, i.e. for any dimension d, the ideas and the steps of the proof are exactly the same as 
for d = 1 but the notations and details of the proof are much more technical. To give an idea, let us succinctly 
describe the ESO relations of arity d+1 introduced in the main normalization step. Here again, each ESO 
relation symbol R of the original sentence <E> in ESO(V rf+1 , arity d+ 1) is replaced by - or, intuitively, "divided 
into" - (d+ 1)! new ESO relation symbol R a of the same arity d+1, where a is a permutation of the set of 
indices [d + 1]. The intended meaning of each relation R a is the following: for each tuple (a\,.. . ,aj+i) e 
[n] d+1 such that a\ < a^ . . . < aj+i, the equivalence 

R a (ai, . . . ,a d +\) <-» R(a a (i), ■ ■ .,a a{ d+\)) 

holds. Then, we introduce a partition of the domain [n] d+l into subdomains, similar to the partition of the 
domain [n] 2 described above for d = 1 into the diagonal x-y and the two "half domains" over and under the 
diagonal x < y and x > y, respectively. According to the case (i.e. subdomain of the partition), this allows to 
replace each R atom in O by an atom of one of the two following "sorted" forms: 

RJx); fl a (x«) 

where x = (xi,...,x^ + i), 1 < i < d+1, and x w is the tuple x where x, is replaced by succ(x ( ). Finally, 
the equalities and inequalities are similarly eliminated in the sentence and we normalize it with respect 
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to the input d-aiy relations (Q s )seX by using new ESO relation symbols of arity d + 1 to convey the input 
information: in the final "sorted" sentence all the Q s atoms are of the unique form Q s {x\,...,Xd). For such 
a "sorted" ESO(V rf+1 , arity d+ l)-sentence <I>, it is now easy to construct a ^-automaton that generalizes the 
automaton described above in case d = 1, and checks in linear time whether coord c/ (p) |= <t>. □ 



Proposition 3.9. For any d>0, ESO(var d) c ESO(V' : ', arity d) on coordinate structures. 

Sketch of Proof. We first prove a kind of Skolemization of ESO(varoT)-formulas, thus providing a first 
normalization of these formulas, in which the first-order part is universal and includes the same number 
of first-order variables than the initial formula. To illustrate the procedure that performs this preliminary 
normalization, let us run it on a very simple first-order formula with two variables: 

if = 3x ( VyU(x,y) V 3yD(x, y) ) . 

We introduce three new relation symbols R\,R 2 ,Ri corresponding to the quantified subformulas of <p. 

defi(fii) = Vjt: Rt(x) <-> VyU(x,y) 
def 2 (fl 2 ) = Vx: R 2 (x) <-> 3yD(x,y) 
def 3 (fl 3 ) = Vx: Rj,(x) «-> R l (x)VR 2 (x) 

Hence, our initial formula can be rewritten: 



3R u R 2 ,Ri 



f\ defi^i) 



Vl<i<3 



A3xR 3 (x). (4) 



It is easily seen that (4) can be written as a conjunction of prenex formulas, each of which involves no more 
than two variables and has a quantifier prefix of the shape VxVy or Vx3y (we include in this latter form the 
subformula 3x Rj(x)). All in all, tp is equivalent to a formula of the form: 

3R U R 2 ,R 3 : VxVyi/f(x,y,R)AVx3ye(x,y,R), (5) 

where i/r and 6 are quantifier- free. In order to put this conjunction under prenex form without adding a new 
first-order variable, we have to "replace" the existential quantifier by a universal one. (Afterward tp, as a 
conjunction of formulas of prefix Vjc,y, could be written under the requisite shape.) To proceed, we get use 
of the arithmetics embedded in coordinate structures. It allows to defining a binary relation W with intended 
meaning: W(x,y) iff there exists z < y such that 6(x,z) holds. This interpretation is achieved thanks to the 
formula: 

f tiling) ^ (W(x,y) ~ G(x,yj) A \ 
,y \ ^mcix(y) ^ (W(x,succ(y)) <^ (0(x,succ(y))\/W(x,y))) J W 

Under assumption (6), the assertion V x3y6(x,y) is equivalent to VxVy : max(y) — > W(jt,;y). This allows to 
rewrite (5), and hence <p, as the formula: 

3^/^3,wf (7) 



VxVy(max(y) ^ W(x,y)) 

which belongs to ESO(V 2 ). 

Thus, the above considerations allow to show the normalization ESO(varaT) = ESO(V'') on coordinate 
structures. It remains to prove ESO(V c/ ) = ESO(V c/ , arity d). It amounts to build, for each formula <t> of type 
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3SSx\,...,0Cd(p, where ip is quantifier-free and R is a tuple of relation symbols of any arity, a formula cD' 
with the same shape, but in which all relation symbols are of arity < d, such that <J> and <t>' have the same 
models, as far as coordinate structures are concerned. The possibility to replace a k-ary (k > d) relation 
symbol R of <1> by li-ary symbols rests in the limitation of the number of first-order variables in <£: each 
atomic formula involving R has the form R(t i ,...,tk) where the f,'s are terms built on x\, . . . ,Xd- Therefore, 
although R is k-ary, in each of its occurrences it behaves as a d-ary symbol, dealing with the sole variables 
X\, . . . ,Xd. Hence, the key is to create a d-ary symbol for each occurrence of R in O or, more precisely, for 
each £-tuple of terms (t\,...,tk) involved in a /^-atomic formula. Let us again opt for a "proof-by-example" 
choice and illustrate the procedure on a very simple case. 

Let $ be the ESO(V 2 , arity 3)-formula 3RVx,ytf(x,y,R), where ip = R(x,y,x) A -^R(y,x,y). Introduce two 
new binary relation symbols R( x ,y,x) and R(y, x ,y) associated to the triple of terms (x,y,x) and (y,x,y) involved 
in O, and fix their interpretation as follows: for any a,b e [n], 

R(x,y,x){a,b) <=> R(a,b,a) and R( y%xy ){a,b) <=> R(b,a,b). 
Then we get the equivalence: 

(S,R) \= Vx,y : R(x,y,x) A ^R(y,x,y) o <S,R> \= Vx,y : R (x , y , x) (x,y) A ^R {y ^ y) (x,y) 

which, in turn, yields the implication: 

S N 3RVx,y : R(x,y,x) A ^R(y,x,y) =>S|= 3RVx,y : R (x , y , x) (x,y) A ^R(y tV) (x,y) (8) 

The converse implication would immediately complete the proof. Unfortunately, it does not hold, since the 
second formula has a model, while the first has not. 

To get the converse (right-to-left) implication of (8), we have to strengthen the second formula with some 
assertion that compels the tuple ^(x,y,.t),^tv,x,y) to be, in some sense, the binary representation of some ternary 
relation. This last construction is more sophisticated than the preceding ones, and we can't detail it here. □ 

4. Hierarchy results 

In Section 2, we have established several natural characterizations of the classes REC (/ in existential 
second-order logic on pixel structures (Theorem 2.6 and Corollary 2.11). Is there similar characterizations 
in coordinate structures instead of pixel structures? One can trivially translate the logical characterization of 
REC' in ESO(V 1 , arity 1) for the pixel representation into a characterization in ESO(V c/ , arity d) for the coor- 
dinate representation. However, one notices that the sentences so obtained in ESO(V c/ , arity d) are "sorted" in 
a sense similar to that evoked in the previous section (see the proof of Proposition 3.8): the atoms involving 
the ESO (resp. input) relation symbols, all of arity d, are of the "sorted" form R(x) or R(x^) (resp. Q s (x)) 
where x is the "sorted" tuple of variables (x\,.. .,Xd) and x' ! \ i e [d], is the same "sorted" tuple where x, is 
replaced by succ(x,)- In fact, one can prove that a li-language L is recognizable iff coord c/ (L) is defined by a 
"sorted" sentence in ESO(V rf , arity d). But requiring the sentence to be sorted seems rather artificial. Can it 
be dropped? The following result shows it cannot. 

Proposition 4.1. For each integer d > 2 and for d-languages represented by their coordinate structures, the 
following strict inclusions hold: 

ESO(V rf_1 , arity d- 1) = ESO(varc/- 1) c REC rf c ESO(vard) = ESO(V'', arity d). 
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That means the following implications hold, 

coord l (L) e ESO(varc/- 1) => L e REC' => coord 1 ^) e ESO(vard), 

few? neither of their converses does. 

The proof of Proposition 4.6 that we skip here uses a simple lemma about the so-called Mirror c / language. 

Definition 4.2. For d > 2, we denote by Mirror^ the set of pictures p : [n] d — > {0, 1} that fulfill p(a) = p(a\2) 
for all a e [n] d . Here, a 12 is the tuple a in which the first and second components have been exchanged (e.g., 
(5,3,4,6)i2 = (3,5,4,6)J. 

Lemma 4.3. For each d>2, we trivially have coord 1 (Mirror^) e ESO(varcf). However, Mirror^ & REC c/ . 

The proof of Lemma 4.3 essentially amounts to a counting argument. It is a simple variant of the proof of a 
similar result in Giammarresi et al. [GRST96] and we skip it. 

Finally, the hierarchy theorem below (Theorem 4.6) involves the number of universal first-order quanti- 
fiers. It uses the following "symmetric" language as a counterexample. 

Definition 4.4. Let symj be the set of pictures p : [n]' 1 — > {0, 1 } that fulfill, for every a = (a\,..., a^) e [n] d : 

1. p(a) = p(a a ) for each permutation a of [d], where a a — (a a (i), a a (d)); 

2. p(a) = p(a sym (i))for all i e [d], where a sym (i) denotes the tuple a whose i' h component a, is replaced by 
its "symmetric value " n + 1 — Of. 

In other words, the values p(a) are defined up to all possible permutations of coordinates and up to all 
possible symmetries a t-> a sym ^y 

Lemma 4.5. For all d>2,we have coord' (sYM d ) e ESO(V'' +1 , arity d) \ ESO(V J , arity d). 

Here is our last hierarchy theorem. 

Theorem 4.6. For each integer d>2 and for d-languages represented by coordinate structures, the follow- 
ing (strict) inclusions hold: 

REC d c ESO(var,i) = ESO(V f/ , arity d) 

tn 

ESO(V rf+1 , arity d) 
in 

NLIN^ ; = ESO(vard+l) = ESO(V rf+1 , arity d+ 1) 
The above strict inclusion ESO(varii) c ESO(V'' +1 , arity d) trivially yields the following result. 

Corollary 4.7. For each integer d>2 and for d-languages represented by coordinate structures, we have 
the strict inclusion: ESO(varii) c ESO(arity d). 

Remark 4.8. Corollary 4. 7 surprisingly contrasts with the equality ESO(var 1 ) = ESO(arity 1 )for d-languages 
represented by pixels structures, see Theorem 2.6 and Corollary 2.11. 



Because of Item 1, Item 2 is clearly equivalent to the same assertion for only i = 1. 



17 



Other hierarchies 

It is also natural to address the question whether the following "hierarchies" about second-order logic 
(SO) over picture languages are strict or not: 

1. Is there a strict hierarchy of SO or ESO according to the second-order quantifier alternation! 

2. Is there a strict hierarchy of the classes ESO(arity d) and ESO(V*,arity d) according to the number of 
ESO relation symbols! 

It is well-known that the answer to question 1 is negative for MSO on word languages and tree languages: 
on these classes of languages, MSO = EMSO holds. That is, the hierarchy collapses at it first level. 

At the opposite, Matz and Thomas [MT97] have answered question 1 positively for MSO over 2- 
dimensional picture languages: the quantifier alternation is strict for MSO on 2-picture languages (in pixel 
representation). We strongly conjecture that their result still holds with a similar proof for any dimen- 
sion d > 2 and the pixel representation. We also conjecture that similar results hold in the coordinate rep- 
resentation: typically, we think that for cZ-languages with d > 2, the quantifier alternation of SO(arityc/) 
gives a strict hierarchy. Unfortunately, we have no idea to prove it, even in the "minimal case", that is 
ESO(arity 2) c SO(arity 2), for 2-languages. 

In contrast, the answer to question 2 is totally known and uniform for picture languages of any dimen- 
sion. In all cases, the hierarchy collapses at its first level. More precisely, Thomas [Tho82] has established 
that every EMSO sentence over words is equivalent to a sentence whose monadic quantifier prefix con- 
sists of a single existential quantifier. Matz [Mat98] has proved the same result over 2-pictures in the pixel 
representation. The proof of Matz can be extended (with slight adaptations) to any dimension d, for both 
pixel and coordinate representations. In other words, in both representations, all the logical classes - essen- 
tially ESO(arity d) and ESO(V^,arity d) - we have studied over picture languages of any dimension are not 
modified by the requirement there should be only one ESO relation symbol. 

Some remarks about the dimensions of pictures 

In this paper, for sake of simplicity and uniformity, we have chosen to restrict the presentation of our 
results to "square" pictures, i.e. pictures of prototype p : [n] d — > S. This may appear as a too strong re- 
quirement. In this section, we explain how our results about logical classes and their relationships with 
complexity classes REC d and NLIN^ can be extended to the "most general" picture languages, i.e. to sets 
of (i-pictures of prototype p : [n\] x ... x [n^] — > Z. "Most general" means as much general as they make 
sense in the logical or complexity theoretical framework involved. 

REC d and logical characterizations in pixel encodings 

All our results about logical characterizations of picture languages in pixel encodings and the class 
REC d , i.e. results of Section 2, hold for pictures of general prototype p : [n{\ X ...X — > S, without any 
restriction, i.e. for all «, > 1, i e [d]. Moreover, our proofs also hold without change, except of course, the 
references to integer n which should be replaced by integer n, according to the involved dimension i e [d]. 

NLIN^, and logical characterizations in coordinate encodings 

The definition of linear time complexity of cZ-automata is not clear for input pictures whose domain has 
the general form [n\] x . . . x [n c /] without restriction. However, linear time, i.e. time 0(n), makes sense when 
the n, are of the same order 0(n), i.e. the pictures are "well-balanced". This justifies the following definition. 
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Definition 4.9. A d-picture language L is well-balanced there is some constant positive integer c such 
that the domain [n\\ X ... X [nj] of each picture p € L fulfills the condition: for i e [d], n < cni, where 
n — max(n\,...,n,i) is called the length of p; we say that the picture p (resp. the picture language L) is 
c-balanced. 

It is straightforward to adapt our notion of linear time to well-balanced picture languages. 

Definition 4.10. A c-balanced d-picture language L belongs to NLIN^ a if there exist a d-automaton J{ and 
a linear function T(n) — c\n + ci such that L is the set of c-balanced d-pictures p accepted by J{ in time 
T(n), where n is the length of p. 

Remark 4.11. Those notions are justified by the following points: 

• The "perimeter" of a c-balanced d-picture p of length n is 0(n) and its size (area, volume, etc., 
according to its dimension d) is \p\ = ®(n d ). 

• So, linear time means time linear in the perimeter of the d-picture p or, equivalently, in \p\ l ^ d . 

We can easily extend all our results about square ^-languages to well-balanced if-languages. In order to 
reduce the well-balanced case to the square case we need some new definitions. 

Definition 4.12. To each d-picture p of length n, one associates its squared d-picture denoted p = of domain 
[n] d obtained by putting the new special symbol □ in each additional cell. Formally, 



Remark 4.13. Let c be a constant. If p is a c-balanced d-picture, then the size of its squared picture p is 



The following result is quite easy to prove. 
Lemma 4.14. Let L be a c-balanced d-picture language. Then L e NLIN'', o Lr e NLIN^,. 

Remark 4.15. Notice that the nontrivial (right-to-left) implication of the Lemma 4.14 means that the d- 
automaton J{\ that recognizes L = in linear time can be simulated by some d-automaton JKi that recognizes 
L in the same time (up to a constant factor) but with less space: only uses the cells of p instead of the 
cells of the squared picture p = . This is possible with the following trick: the fact that the i th dimension n of 
p = is replaced by nj >c~ l n allows to "fold" c times the picture p = along the i th dimension nj of p. All in all, 
each cell of p simulates (at most) c d cells of p = . This is performed by taking for the set of states of ^2 the 
set r c where T is the set of states of 

Now, let us compare any well-balanced li-picture language L and its squared language Lr from a logical 
point of view. The domain of the coordinate representation of a li-picture p : [n\] X ... X [n<j] is naturally [«] 
where n is the length of p, i.e. n = max(«i , . . . , n c i). So, we define the coordinate representation of p as 




p(a) if a € dom(p); 
□ otherwise. 



To any d-picture language L one associates its squared d-picture language 



L ={p :p€L}. 



\P = \ = ®(\P\) 



coord rf (p) = coord rf Q? ). 
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Hence, as a trivial consequence of the definitions, 

coord c/ (L) = coord d (Zf). 



The previous definitions and remarks and Lemma 4.14 justify that the results of Sections 3 and 4 hold 
without change in the extended case of well-balanced cZ-picture languages. 



Conclusion 

Notice that Theorem 3.7 that characterizes the linear time complexity class of nondeterministic cellular au- 
tomata is very similar to the following result about time complexity 0(n d ), for any d > 1, of nondeterministic 
RAMs, by two of the present authors [GO04]: 

NTIME ram (n i/ ) = ESOF(var£f) = ESOF(V rf , arity d). 

The main difference is that this latter result involves the existential second-order logic with functions (ESOF) 
instead of or in addition to relations and holds in all kinds of structures without restriction: pictures, struc- 
tures of any arity and any type, etc. It is also interesting and maybe surprising to notice that, in those results, 
the time degree d of a RAM computation plays the same role as the dimension d + 1 of the time-space 
diagram of a linear time bounded computation for a af-dimensional cellular automaton. 

Both results attest of the robustness of the time complexity classes NTIME lam (« rf ) and NLIN^. They 
stress the significance of the RAMs as a sequential model, and of the cellular automata as a parallel model. 

Finally, if we continue the comparison, we have, for coordinate representation of li-languages and for 
d>2, the (strict) inclusions: 

REC rf c ESO(V rf , arity d) c ESO(V rf+1 , arity d) c NLIN^ a 
for cellular automata and ESO, which strongly contrast with the equalities: 

NTIME ram (« c/ ) = ESOF(V', arity 1) = ESOF(V 2 , arity 1) 
for RAMs and ESOF [GO04, DGO04]. 
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Appendix A. Normalization of "guessed" relations 

This appendix is dedicated to the proof of the normalization of ESO(V'', arity d) on coordinate encodings 
of (d- l)-pictures, for d>2. Our purpose is to rewrite ESO(V c ',arity cf)-formulas into equivalent formulas 
that fulfill the "sorted" or "local" property mentioned in the proof of the "sufficient condition" of Proposi- 
tion 3.8. 

Before formalizing our notion of "sorted" formulas, let us detail what is its intended meaning. We want to 
deal with pixels of time-space diagram of the computation that are adjacents, that is, that are both connected 
and differ (by one) by at most one dimension. Two such pixels are represented by of-tuples of the form x and 
x®, that is: 

• their components are in the same order (elsewhere they could be disconnected); 

• there is at most one occurrence of succ (elsewhere, they would differ of more than one dimension). 
These requirements (and a little more) are formalized in the following definition. 

Definition Appendix A.l. Let k, d be two integers such that d>k> 1. A sentence over coordinate structures 
for k-pictures is in ESO(V (/ , arity d, sorted ) if it is of the form 3RVxi^(x) where 

• R is a list of relation symbols of arity d; 

• ip is a quantifier-free formula whose list of first-order variables is X = (x\ Xd); 

• each atom of\p is of one of the following forms: 

(0 Qs(xi,...,Xk),for s eS, 

(ii) R(x) or R(x^) where R 6 R, i e [d], and x^ is the tuple x where Xj is replaced by succ(xi), 
(Hi) min(xi) or max(xi), for i e [d]. 

We prove the normalization ESO(V f/ , arity d) = ESO(V rf , arity c/, sorted) for (d— l)-pictures (i.e., for k = d — 
1). In the present section, we deal with Condition (ii) of the above definition (see Proposition Appendix A. 9). 
In Subsection Appendix A.l, we eliminate equalities and inequalities. At this point, we get a normalization 
of ESO(V'', arity d) into the so-called "half-sorted logic", denoted by ESO(V rf , arity d, half-sorted ). It remains 
to manage the input relation symbols; this is done in Section Appendix B, where we tackle Condition (/). 

To lighten the presentation of the forthcomming results, we first introduce some notations about tuples 
and permutations. 

Definition Appendix A.2. Let n,d>0 and x e [n] d . 

• We denote by [x] ; - the i' h component ofx. E.g. (5,7,2)2 = 7. 

• We say that x is non decreasing, and we write xT, when [x] i < ■ • ■ < [x] c i. 

• S(d) stands for the set of permutations of{\,...,d). Given a\,...,ad e {l,...,d}, we denote by a\ ...ad 
the permutation a e S(d) that maps each i on a/. Conversely, for a e S(d) we set a-,- :— a(i). By T(d) 
we denote the set of transpositions over {l,...,d}. 

• If a € S(d) and x — (x\,...,Xd), we denote by x a the d-tuple (x ai ,...,X ad ). It is less ambiguous to 
define x ff by the assertion: 
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foranyie]!,..., d], [x a ] t = [x] a(i) . 

Thus, if fj also belongs to S(d), we get [(Xq.)^],- = [%a]p(i) = Wafi(f)- Whence the identity: (x a )p = x a p. 
In particular, (x a ) a -i — x. 

• For a e S(d) and n > 0, we set [a] = {x e [n] d s.t. x a 1}. In particular, denoting by id the identity on 
{l,...,d}, we get x e [id] iff*]. Therefore, x e [a] iffx a e [id]. Clearly, [n] d = (JasS(d)M- 

• For any i in { 1 , . . . , d], X® denotes the tuple obtained from x by replacing its i ,h component by its own 
successor. That is, ifx — (jci, . . . , xj) then 

X® = (xi, . . . ,Xi-i, succ(xj), . . . , Xd). 

As previously, the arrangement o/x® according to some permutation a is denoted by (x®)^. 

Example. Consider x = (5,3,7,2) in [9] 4 and a = 4213, /3 = 1432 in 5(4). Then x a = (2,3,5,7) is non- 
decreasing while x p = (5,2,7,3) is not. Besides, x (3) = (5,3,8,2) while (x a ) 0) = (2,3,6,7) and (x (3) ) ff = 
(2,3,5,8) = (x ff )< 4 >. 

With these notations, the request described at the beginning of the section can be rephrased as follows: 
we want to normalize ESO(V c/ , arity af)-formulas in such a way that each atomic subformula R(t i (x), . . . , t p (x)) 
built with a guessed relation symbol R has either the form R{x) or the form R(x^). 

Fact Appendix A.3. On coord'' -1 , 5 any formula O = 3RVx^(x, R, <x) e ESO(V'', arity d) of signature o~ can 
be written in such a way that: 

(a) In each atomic subformula R{t\,. . .,t p ) of ip, Var(tf) Ci Var(tj) — for every 1 < i < j < p. 

(b) Each R in R has arity d exactly. 

Proof. The proof of (a) is quite immediate. We illustrate it with an example: assume that <& involves the 
subformula 7?(succ 2 xi,X2,succxi,succ 3 X2) for some R e crU{R). Then clearly, <D is equivalent, on picture- 
structures, to: 

BRBAVx : (jo(x,R,A,cr) A (x\ = SUCCX3 f\x^ = succ 3 ^) — > (A(x2,xt,) <-» R{x\,X2,xt,,X4)) 
where tp is obtained from ip by substituting the formula A(x2,succ(jci)) to each occurrence of the atom 

/?(SUCC 2 X1 , X2, SUCC X\ , SUCC 3 X2). 

In order to prove (b), assume for simplicity that R reduces to the single relational symbol R of arity 
p < d. The idea is to add d-p dummy arguments to R. Clearly, <& is equivalent to the formula: 

3R : ip A Vx f\ p<i < d R(xi , . . . ,x t , . . . ,x d ) <-> R(x u . . . , succ(x ; ), . . . , x d ), 

where ip is obtained from (p by replacing each atomic subformula R(t\,...,t p ) by R(t\,...,t p ,Xi 1 ,...,Xi d ). 
Here, x^,..., Xi d is the complete list of distinct variables among x that do not occur mt\,...,t p . □ 



5 We denote by coord'' 1 the sets of coordinate encodings of (d- l)-pictures. 
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Remark Appendix A.4. The proof of Fact Appendix A.3 allows enhancing its statement: each atomic sub- 
formula of '(p that involves an input symbol Q a , for some aeS, has the form Q a (x ai , . . .,x adl ) where a is an 
injection from {1 , . . . , d — 1} into {1, . . . , d}. 

Fact Appendix A.5. On coord 6 ' -1 , any formula <D = 3RVx(,o(x, R, cr) e ESCKV^arity d) of signature cr can 
be written in such a way that each atomic subformula over R ofip has one of the two forms: R(x^) or R(x„), 
where n e S(d). 

Proof. We prove the result for d — 3. The general case is similar. Let I be the maximal value of an ; e N 
such that succ'(x) occurs in O, for any x ex. For each R e R, we introduce new cZ-ary relation symbols Rijjc 
for every i,j,k< I. We want to force the following interpretations of the Rijjc's: 

Ri ij c {u\,U2,Uj) - /?(sUCC ! Mi,SUCC' / M2,SUCC i: M3). 

This is done inductively, with the formulas: 

• Vx : #o,o,o(*i . xi, x 3 ) <-> R(x\ ,x 2 , x 3 ) 

• Vx:/\ /\(Ri+l,j,k{Xl,X2,X3)<r+Rij tk (s\XCC(Xl),X2,X3)) 

l<t j,k<{ 

• VX:/\ f\(Ri,j+l,k(Xl,X2,X3) ^RiJ,k(Xl,SMCC(x 2 ),X3)) 

j<( i,k<t 

• Vx:/\ f\(Ri,jjc+l( x ux 2 ,X3) <*R i jj c (xi,x 2 ,suce(x3))) 

k<( i,j<( 

Factorizing the quantifications and using notations of Definition Appendix A. 2, the conjunction of these 
formulas can be written: 

#o,o,o(x) «-> R(x)A 
/\ /\(/? I+ ij,fc(x)^^(x (1) ))A 

i<C j,k<t 

f\ f\(Ri, ]+ u{x)^R^ k (x {2) ))A 

j<l i,k<t 

f\ /\(R UM i(x) " Ru,k(x 0) )) 

Let us denotes by decomp(7?, (Rijjdi,j\k<{ ) this last formula. It clearly fulfills the condition of the statement. 
Now, consider the formula 



Vx 



3R3((Rij,k)ij,k<e))ReR ■ /\decomp(R,(R idk ) idk < e ) A Vx£, 



(A.l) 



ReR 



where ip is obtained from ip by the substitutions 

R(succ'x ai , SUCC-'Xq.j , SUCC^Xaj) Ri,j,k(x ai , X a2 , Xa 3 ). 

Then, the formula (A.l) is equivalent to <I> and also fits the requirements of Fact Appendix A. 5. It is the 
rewriting of O announced. □ 
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As a result of Fact Appendix A. 3, Remark Appendix A. 4 and Fact Appendix A. 5, each ESO(V rf , arity d)- 
formula of signature cr has a conjonctive normal form of the shape: 



<D = 3RVx/\ \J 




! 



Furthermore, the trick used in the proof of Fact Appendix A. 5 also allows writing atoms involving min, max 
or equalities, under the form max(x), min(x) and x = y for some x,y ex. 

It remains to prove that we can get rid of the atomic formulas R(xp), where ft + id. This part is rather 
technical, so we provide some preliminary explanations before stating the logical framework which allows 
the normalization. In order to get rid of each literal of the form R(xp), we will divide the set R c [n] d in d\ 
relations Rp c [n] d , each corresponding to a given permutation (3 of {1, . . . ,d}. 

Definition Appendix A.6. For R c [n] d and for each a e S(d), we define a d-ary relation R a on [n] by: 
R a = {x e [id] s.t. R(x a -i )}. Alternatively, R a can be defined by: R a = \x a : x € R n [a]}. 

Thus, Definition Appendix A. 6 associates with each R c [n] d a family (R a )aeS(d) of relations, each of 
which is entirely contained in the set [id]. This family is intended to represent R through its d\ fragments 
according to the partition [n] d = UoeS^M- Namely, each R a encodes the fragment R n [a] over [id]. 

Actually, \J ae sid) M is not really a partition, since the [a] 's can overlap. Hence, Definition Appendix A. 6 
induces some connexions between the relations R a : if some x is both in [a] and in \J3], or equivalently, if 
x a = xp, then x e R n [a] iff x e R n \J3] and hence, by Definition Appendix A. 6: R a (x a ) = Rp(xp). We will 
keep in mind : 



The following lemma states that condition (A. 2) ensures that the R a 's issue from a single relation R, 
according to Definition Appendix A. 6. Besides, a new formulation of the condition is given in Item 3 of the 
lemma, that will better fit our syntactical restrictions. 

Lemma Appendix A.7. Let (Ra)aeS(d) be a family of d-ary relations on [n] such that R a c [id] for each a. 
The following are equivalent: 

1. 3R c [n] d such that R a = {x e [id] s.t. R(x a -i ))for each a € S{d) ; 

2. Vq-,/3 e S(d), Vx e [n] d : x a =x p ^ R a {x a ) = Rp(xp) ; 

3. \/a e S(d), Vt e T(d), Vx e [n] d : x = x T R a (x) = R aT (x). 
(Recall T(d) denotes the set of transpositions over {l,...,d}.) 

Proof. 1 => 2: See Equation (A. 2). 

2 => 1: For (R a )aeS(d) fulfilling 2, consider the relation R C [n] d defined by: 

R(x) iff R a (^a) for some a such that x a t ■ (A.3) 

This definition is well formed, since for any a, j3 e S(d) and any x e [n] d such that both x^T and Xgt hold, we 
have x a = xg and thus, by 2, R a (x a ) - Rp(xp). Now, let a e S(d). For any x e [id] we have (x^-Oat (since 
(Xtf-Ocr = x) and hence, by (A3), R a (x) = R a ((x a -i)a) = R(x a -i)- Besides, R a (x) = for any x £ [id], since 
Ra £ [id]- Thus R a is obtained from R as required in 1. 



Va,jS G S(d),Vx e [n] d :x a =xp^ R cr (x a ) = Rp(xp). 



(A.2) 
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2 => 3: Let a e S(d), r € 7~(t/) and x e [n] d such that x = x T . Set y = x a -\ . Then, y a — x = x T = (y<*) T = y ar . 
Therefore we get by 2: R a {Sa) = Rarfyar), an d hence: /? a (x) = R aT (x). 

3 => 2: Let a,/? e S(d) and x e [n\ d such that Xq. = xp. For y = x„, the equality x a = xp can be written 
y = y a -ip- It means that the permutation a~ l /3 exchanges integers that index equal components of y. It is 
easily seen that this property can be required for each transposition occuring in a decomposition of a~ l /3 
on T(d). That is, there exist some transpositions ti,...,t* € T(d) such that a~ l /3 = T\ . . .tj and y = y Tl = 
Ynn = •■• = Yri,..T r T hen , applying 3 to these successive tuples, we get: R a (y) = R aTl {y T i) = Rarm^rm) = 
■■■ = RaT l ...T k (y Tl ...T k )- Hence R a (y) = Rp(y a -ip), that is R a (x a ) = Rp(xp), as required. □ 

Lemma Appendix A.8. Let R and (Ra)aeS(d) b e defined as in Definition Appendix A.6. Let a,/3 e S(d) and 
i e {l,...,d}. For any x e [a]: 

1. R(xp) is equivalent to Rp-i a (x a ). 

2. R(x^) is equivalent to: 

(xi = X ad AR aTa _ Ht)d ((X a )W)j V \J (xi = X ak < x at+l AR aTa _ lm ((xJ k ^. 

i<k<d 

Proof. 1. We have seen that R(x) holds iff R y (x y ) holds for any y e S{d) such that x r f. Thus we get, for a 
given p e S(d): R(xp) iff R y ((xp) y ) holds for any y e S{d) such that (xp) 7 1. That is, since (xp) 7 = xp y : 

R(xp) iff R y (xp y ) holds for any y e S(d) such that x^f- (A.4) 

In particular, xp y 1 iff xp y = x a , since x e [or]. Thus, y = /3~ l a is one of the permutations such that Xpy\. Thus, 
replacing y by f3~ l a in Equation (A.4), we get the sought result. 
2. From x® = (xi,..., succ(x,), . . . , Xd) we get: 

)a — i. x a\ » •• ■ >%ai-i » SUCC(Xj), X^j +i , . . . ,X ffrf ) 

where y = a~'(0- Since x ai <■■■< x a ., the above tuple (x®) a is almost increasingly ordered. More precisely, 
there exists k e {1, . . . , d] such that: 

where y = aT 1 ^'). Clearly, the largest such k is characterized by: (k = d) or (k < d and xi = x ak < x ak+l ). Or 
equivalently, by: 

(Xi - x ad ) ov(k<d and x t = x ak < x aji+l ). (A.5) 

If we denote by Tp the transposition over { 1, . . . ,d} which permutes j and k, the definition of k yields that 
the tuple 



ix^'^ajj^ — (x a ^ , . . . , X a j_i , X(y k , x a ^ , . . . , x ak _^ , succ(jc/) , x ak+ ^ , . . . , x ad ) 



is non decreasing. Hence, R(x^) = R aTjk {{^) a r jk ). Besides, since = xu the tuple above can 

also be written: 



(x )qtj£ — (-^ari j • ■ • 5 X(x-\ , X/ , X^-^p . . . , Xq,^ ! , SUCC(Xq^) )X(x k+ ^ , . . . , Xa^) 
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That is: {x (i) ) aT . k = (x a ) (k \ Therefore: #(x®) = R aT - k ({x a ) ik) ). Reminding that j = a~\i), we can finally 

state: there exists a sole k 6 {i d] defined by (A. 5), and for this k we have: R(x^) = R aT , ((x») )• 

The conclusion easily proceeds. □ 



Proposition Appendix A.9. Ford> 1, ESO(V (/ ,arity d) c ESO(V'',arityt/, half-sorted) 0« COORD' 



Proof. To simplify, assume we want to translate in ESO(V c/ ,arity ii, half-sorted ) some ESO(V rf ,arity<i)- 
formula of the very simple shape: = 3RVx(p(x,R), where R is a (single) ii-ary relation symbol, x is a 
(f-tuple of first-order variables, and <p is a quantifier-free formula. Since the sets [or], a e S(d), cover 
the domain [ri], we obtain an equivalent rewriting of <S) with the following artificial relativization: <D = 
B/JVx/v^g^^/j (x e [a] — > tf). Furthermore, all atomic subformulas of tp built on R can be assumed of the 
form ^(xg) or R(x (l> ), thanks to Fact Appendix A. 5. 

To get rid of these literals, we substitute to R a tuple of relations (R a )a€S(d) that encode R on the sets 
[a]. Recall we proved in Lemma Appendix A. 7 that this substitution is legal as soon as R a c [id] and 
/? a (x) = RaAx) for all a e S(d), t e T(d) and every x e [n] d such that x r = x. Then, Lemma Appendix A.8 
gives the translation of /^-atomic formulas into formulas expressed in term of the R a 's. All in all, we get the 
equivalence of the initial formula <D to the following: 



Vx f\ (RJx) -» x e [id])/ 



aeS(d) 



Vx f\ f\ (x T = x^(R cr (x)~R aT (x)))A 

aeS(d)T€T(d) 

VX A ( X 6 ^ ~* <Pa(x,(Ry) 7 eS(d))) 
a€S(d) 



(A.6) 



where each tp a is obtained from if by the substitutions: 

(x i = x ad AR aTa _ lm «xJ d >)} v 



R(x (i >) ■ 



i<k<d 



*LY k <x a k+ i A ^»T a _ l(j)Ji ((x t 



«)*>)) 



The first two conjuncts of (A.6) ensure that the family (R cr )a€S(d) encodes a relation R (see Lemma Ap- 
pendix A. 7) ; the third interprets assertions of the form R(x/j) and R(x^) according to the modalities de- 
scribed in Lemma Appendix A.8. Because of permutability of the conjunction and the universal quantifier, 
this third conjunct can be rewritten: 



/\ Vx : x e [a] -h> ipa(x,(R y )y€S(d)) 



(A.7) 



aeS(d) 



For a fixed conjunct in (A.7), i.e. for a fixed a, all atomic subformulas of tp a built on the R 7 's have the 
form R y (x a ) or R y ((x a )^) for some y e S(d) and some i e{l,...,d}. Hence, the substitution of variables x/x a 
allows to write such a conjunct as: Vx : x € [id] — » <p a where <p a = ip a (x/x a ) only involves (/? r )-subformulas 
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of the form R y (x) or R y (x (k) ) for some y e S(d) and k e {1 , . . . , d). Finally, the initial formula <& is proved 
equivalent to: 



' Vx /\ (R a (x) -» x e [id])/ 



Vx /\ /\ (x T = x -» (/? a (x) «-» fl OT (x)))A 
Vx /\ (x e [id] -> £ a (x,(fl 7 ) r&S(rf) )) 



that fulfills the requirement of Proposition Appendix A.9. 



Appendix A.l. Getting rid of arithmetic 

Does mean introducing new second order variables of arity 2, we can assume that our formula <£ e 
ESO°"(V rf ,arity d) - where each relation symbol of arity d < 2 occurs in normalised (i.e. sorted) form - 
involves no comparison (equality, inequality) relation. We obtain that in two successive steps. 

First, if €> involves inequalities < and >, then it is equivalent to the following formula <£' without in- 
equalities (but with equalities) and two new binary relation symbols < and >: 



O' = 3 < 3 >: O A Vxi , X2 



(x\ — X2 V xi < xj) 
{x\ — X2 V x\ > xj) 

X\ < X2 
X\ > X2 



(-imax(^2) — * x\ < succ(jc2)) A 

(-imax(xi) — » succ(xi) > X2) A 

(i(xi > X2) A -i(jq = X2)) A 

-.(JCI = x 2 ) 



where 3) is obtained from <I> by the substitutions u < v u < v (resp. u > v ™» u > v). This is justified 
as follows: the first two conjuncts of the subformula VxiV^I' • • } express that x\ < X2 x\ < X2 (resp. 
x\ > X2 => x\ > X2). The third and fourth conjuncts express that the three relations <,> and = are totally 
disjoint. That implies that < and > have their exact meaning. Notice also that the occurrences of < and > 
preserve the sorted property. 

Secondly, if <S> involves equalities (without any inequality), it is equivalent to the following formula <t>', 
written without the symbol "=" but with the new binary symbol »: 

O' e 3 »: <J> A \/x 1 ,x 2 x ¥ 

where 4> is obtained from <£ by replacing each equality u — v by u x. v, and *F is the conjunction of the 
following formulas: 

• min(xi) — > (min(x2) *-* X\ » X2); 

• min(x2) — > (min(xi) «-> x\ « X2); 

• (-imax(xi) A -imax(x2)) — > (x\ « X2 <-> succ(xi) « succ(x2)). 

Notice that this transformation preserves sorted property. The results obtained until now can be recapitulated 
as follows: 
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On (d — l)-pictures, each ESO cr (V c/ ,arity af)-formula can be written under the form: 

a -I™, A \ / f min(^), max(x), 1 D ^ 
* S 3RVx/\V±{ fi(Xi) , fl (x) , fl (x (0) ) (A.8) 

Here, R (resp. Q) is a list of relation symbols of arity d (resp. d— 1), x e x, Q e U(<2 a )ae£» ReR, 
i e J(d), / e {l,...,d}. 



Appendix B. Normalization of input relations 

Let's come back to the normalization of the formula O of Equation (A.8). For simplicity, assume that it 
has the restricted form i> = 3RVx</>(x,R, Q,cr), with only one relation symbol Q of arity d— 1. 

We aim at defining a tuple of relations (Q a )a<Ei(d), m sucn a wa y that 2«( x ) = 6( x ») f° r eacn x - Clearly, 
such relations will allow to write O under the desired form, by replacing each subformula Q(x a ) by the 
sorted translation: Q a {x). The difficulty is to express this definition with our syntactical restrictions, that is, 
without involving any x a with a + id. 

Notice that the strategy used in Section Appendix A to "sort" atomic subformulas R(x a ) build on any 
existentially quantified cZ-ary relation R is no more avalaible, since it means suppressing R in favour of some 
new existentially quantified relations. Of course, we can't operate like this with the input relation Q. To give 
an hint of the method used in this section, let us consider an easy example. 

An easy case.. Consider the case where d -2. We deal with two first-order variables x and y and we only 
accept atoms of the form Q(x), R(x,y), R(succ(x),y) and R(x, succ(y)) for any input unary relation Q and any 
guessed binary relation R. How can we tackle occurences of some atom Q(y) in the formula ? A natural idea 
is to define a new binary relation Qi in such a way that 22(^,3') = Qiy) holds for any x,y. (We denote it Q2 
to refer both to Q and to the projection of (x,y) on its second component.) Hence, we set: 

Qi = {(x,y) : Q(y)}. 

Thus, any atom Q(y) could be replaced by the sorted atom Qiix^y). But the logical definition of Q2 with 
our syntactical constraints compells to introduce an additive binary relation T that will be used as a buffer to 
transport the information Q(y) into the expression Q2(x,y). We set 

T = {(x,y):Q(x + y)}. 

Clearly, T is inductively defined from Q by the assertions T{x,Q) = Q{x) and T(x+l,y) = T(x,y+ 1). Besides, 
Q2 is defined from T by (22(0, y) = T(Q,y) and Q2(x,y) = Q2(x+ l,y). All these assertions can be rephrazed 
in our logical framework, with the following formulas: 



Vx,y 



min(y) -> (T(x,y) «-> Q(x)) A 
(-imax(i) A -imax(y)) — > T(sncc(x),y) «-» T(x, succ(y)) 

min(x) -» (Q2(x,y) <^ T(x,y)) A 
Q 2 (x,y) o g 2 (succ(x),y) 



Now, it remains to insert this defining formulas in the inital formula O to be normalized, and to replace each 
occurence of Q(y) by Q2(x,y). Of course, such a construction has to be carried on for each input unary 
relation Q. 
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The general case.. Given i,j 6 {l,...,d} and a e S(d), we denote by (if) the transposition that exchanges i 
and /', and by a(ij) the composition of a and (if). It is well-known that each permutation a can be written as a 
product of transpositions, a = (u\ v\)(u 2 v 2 ) ■ ..(upVp). It is easily seen that this product can be chosen in such 
a way that v; = Uj+\ for any i. This is because if some sequence (ab)(cd) with b + c occurs, it can be replaced 
by (ca)(ab)(bc)(cd), and a well chosen iteration of such rewritings yields the desired decomposition. This 
can be further refined, by fixing at d one element of the first transposition involved in the decomposition and 
by prohibiting useless sequence as (ab)(bd). Finally, each a e S(d) can be written 

a = (du\)(u\u 2 ) ■ ■ .(u k - 2 u k -i)(u k -\u k ), (B.l) 

where m+\ and Uj +2 are pairwise distinct elements of [l,...,d] for any i. We call alternated factorization 
of a such a decomposition. 

A permutation a admits of several alternated factorizations, and we want to single out one of them for 
each a, in order to allow an inductive reasoning build on this particular decomposition. There is no canonical 
way to perform this task. In the following lemma, we roughly describe one possible choice, that implicitely 
refers to the graph on domain S(d) whose edges correspond to those pairs of permutations (a,j3) such 
that fj - a(if) for some i, j s {1, . . . ,d). 

Lemma Appendix B.l. There exists an oriented tree 7~j covering S(d) which is rooted at id and such that 
each Td-path starting at id, say ida\ . ..a p , corresponds to an alternated factorization of a p . 

Proof. Trees T<i for d > 2 are defined inductively. For d = 2, there is a unique such tree: (12) — » (21) = 
(12)(21). So, assume we are given T&-\ and carry on the construction of T c i as follows: 

(a) First, view each permutation a - a\ ... a c i-i e S(d - 1) as a permutation of {2, . . . , d] by renaming a,- as 
a; + 1. That is, replace a — a\ . ..ad-i by a + = (ai + 1) . . . (ad-\ + 1). 

(b) Then, replace each such a + by fj — f3\ ...fid £ S(d) defined by: f3\ — 1 and fJi = [ar + ];-i = + 1 for 
i > 1. Thus, Td-i now covers the set of permutations fj e S(d) that fulfill = 1. 

(c) For each node fj thus obtained, create a new node labelled by the composition of f3 with the transposition 
(Id) - that is by the permutation fJ(ld) - and create an edge fj — > fj(ld). 

(d) Finally, link each such node fJ(ld) to d-2 new nodes fj(ld)(di), for i = 2, . . .,d— 1. 

In Fig B.l, we display the steps of the construction of T4 from T 2 . Letters (a), . . . , (d) in the figure refer to 
the above items. The correction of the method on this example is clear. We leave it to the reader to verify 
that it generalises to any d. □ 

This lemma allows us to choose, for each a € S(d), one alternated factorization of a: it is the de- 
composition (dii)(iii 2 ) ■ ■ - (i k -ii k ) corresponding to the unique path from id to a in T c \. We denote by 

id.di\.i\i 2 i k -\i k this particular factorization. And when this path until a can be continued in T c \ to 

some permutation a(i k i k+ \), we denote by a.i k i k+ \ this last permutation. For instance, in the example dis- 
played in Fig B.l, we can write 2143 = 4123.13 and 3124 = 4123.14 while 4321 = 4123(24) cannot be 
written 4123.24. Notice furthermore that the integers i k and i k+ \ are ordered in the notation a.i k i k+ \ (unlike 
in the notation a(i k i k+ \)): we place in first position the integer i k involved in the last transposition leading 
to a (with i k — d if a — id). All in all, the reader is invited to keep in mind the numerous presuppositions 
attached to the notation a.uv: the statement fj = a.uv means: 

• fJ — a(uv) ; 
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(b) 123 «- 132 



(c) 321 -« 123 - 132 312 

(d) 231 -> 321 - 123 132 - 312 »■ 213 

(a) 342 •* 432 -> 234 " 243 423 »- 324 

(b) 1342 1432 1234 1243 " 1423 1324 

1342 " 1432 ■+ 1234 — 1243 — 1423 " 1324 

(0 I I I I \ I 

4312 4132 4231 4213 4123 4321 

1342 1432 ■« ( 1234) 1243 1423 1324 

\ \ \ \ \ \ 

W 4312 4132 4231 4213 4123 4321 



/\ /\ /\ /\ /\ /\ 

2314 3412 2134 3142 2431 3241 2413 3214 2143 3124 2341 3421 

Figure B.l: Construction of T4 from 7~2- The result is an oriented tree rooted at id, spanning <S(4), whose all pathes from 
id are alternated. 



• (a, (3) is an edge of T d ; 

• either a — id and u = d, or a — y.tu for some y € S(d) and some t ± u in {1, .. ,,d}. 

Let us now come to a straightforward remark connecting c/-tuples to alternated factorizations. First recall 

that for x = (xi,.. -,Xd), i e {1 d) and a,/3 e S(d), we denoted by [x],- the / th component of x, we defined 

x a as the of-tuple (x ai , x ctd ), and we noticed that (x a )p = x a p. (See Definition Appendix A.2.) 

Fact Appendix B.2. For any x e [n] d , a e S(d), and i, j 6 {1, . . . ,d): 

(a) X a jj = (X a \ij). 

(b) [X a jj] j = [x] d . 

Proof, (a) From a.ij = a(ij) and x a p = (x a )fj. (b) By induction on a: if a = id, then necessarily i = d and 
[ x id.c/;]_/ - Mrf clearly holds. Otherwise, \x a .ij\j = l( x a)ij]j = [x a ]i = [x] c i, by induction hypothesis. □ 

Given a cZ-tuple x = (jci, . . . , x c [) of first-order variables, we denote by x~ the (d - 1 )-tuple obtained from 
x by erasing its last component. That is, 

(xi,...,x d -i,x d )~ := (xi,...,Xd-i). 

In particular, for a e S(d), we denote by x~ the (d- l)-tuple (x ai , . . .,x ad l ). Each (d- l)-tuple build upon the 
d variables x\,...,x d can clearly be written x~ for some a e S(d). Therefore, the occurence of a non-sorted 
atom in O has the form Q(x~) for some Q e (2a)«ei and some a e S(d), and the purpose of this section 
amounts to rewrite each such occurence Q(x~) as Q'(x) for some well chosen relation Q'. 
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Definition Appendix B.3. Given a (d— \)-ary relation Q and two family ofd-ary relations, (T a ) ae s(d) an d 
(Q(i)aaS(d)> we say that (T a , Q a ) a eS(d) !i cZ-simulation of Q if the following axioms hold, for any a e S(d) 
and any i,j < d such that a.ij is defined, and for any d-tuple x of variables: 

(AO T id (x) = Q(x-)if[x] d = 0. 

(A2) T a _ij(x) = T a jj(X(ij)). 

(A 3 ) T a (x) = T a ,ij(x) if[x]i = 0. 

(A 4 ) GjhOO = g(x-). 

(A 5 ) Qa.ij(x)=T a .ij(x)if[x\j = 0. 

(Ae) Qa.ij(x) doesn't depend on [x]j. 

Lemma Appendix B.4. Let (T a ,Q a ) ae s(^ be a d-simulation of some (d — \)-ary predicate Q. For any 
x e [n] d and a € S(d): Q a (x a ) = Q(x~). 

Proof. Let us first prove that for any x € [n] d and a e S(d): 

[x] d = => T a (x a ) = g(x-). (B.2) 

We proceed by induction on a. If a - id, {B.2) follows from (Ai). Given a non-identique permutation 
a.ij, we have: 

Ta.ij(x a .ij) = Tajjdxa)^) by Fact Appendix B.2-(a). 
= T aAj (x a ) by(A 2 ). 

But [x a ]j = [(x a )(,/)]i = [x a ,ij]j and hence, by Fact Appendix B.2-(b): [x a ]j = [x]^ = 0. Therefore we can 
resume the above sequence of equalities with: 

T a .ij(x a .ij) = T a (x a ) by (A 3 ) since [x ff ],- = 0. 

= 2( x ~) by induction hypothesis. 

This completes the proof of (B.2) 

Let us now prove the equality Q a (x a ) = Q(x~). If a = id, the result comes from (A4). For a non-identique 
permutation a.ij, we have to prove Q a .ij(x a .ij) = Q(x~) for any tuple x € [n] d . First notice that we can 
restrict, without loss of generality, to the case where [x]^ = 0. Indeed, we can otherwise consider the tuple 
y obtained from x by setting [x]^ to 0. (That is, y only differs from x by its d th component, which is null.) 
Clearly, y~ = x~. Besides, the j th component of X a ,y is [x]d, from Fact Appendix B.2-(b). Similarly, the j ,h 
component of y^ ;/ is [y]^. Hence, the tuples x a n and y^;; coincide on each component of rank distinct from 
Therefore Q a .i]{x a .if) = Q a .ij(j a .ij) by (A 6 ) and we get: Q„jjix„jj) = Q(x~) iff Q a .ij(j a .ij) = GOO- Thus, 
we can assume [x]j = 0. It follows [x a n]j = [x]d = 0, by Fact Appendix B.2-(b), and hence: 

Qa.iMa.ij) = T aA j{x aA j) by (A 5 ), since [x ff . i; ] j = 0. 
= Q(x~) by (B.2), since [x] d = 0. 

The proof is complete. □ 
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Lemma Appendix B.5. Let Q be a (d— \)-ary relation and (T )a&S(«i)< (Qa)aeS(d) be two tuple of d-ary 
relations satisfying, for each d-tuple x = (xi , . . . , xj): 



Then (T a , Q a ) a aS(d) is a d-simulation of Q. Furthermore, each Q admits such a d-simulation fulfilling 



Proof. Clearly, the formula (F,) is a mere transcription of the axiom (A,) for each i + 2. We have to prove 
that (F2) implies (A2). Formula (F2) yields that T a jj(x) has the same value on tuples of the form 

x = (u,x + c,\,y-c,w), 

for any c < min{n - 1 - x,y}, where x + c (resp. y-c) is the component of rank i (resp. j) of x. (That is: 
u e [n] !_1 , v 6 and w e [ri\ d ~K) In other words, the value of T a ij(x) depends on [x], + [x] ,• rather than 

on the precise values of these two components. As a consequence, for any u e [re] , v e [n]j~ , w e [n] d ~i 
and any x,y € [n]: 

T a .ij(u,x,\,y,-w) = T a .ij(u,y,v,x,yr). 

This is Axiom (A2). 

It remains to prove that such relations (7V)a&S(<i) an d (Qa)aeS(d) exist for every Q. To see this, assume 
we are given a (d- l)-ary relation Q and define the Qa's, and the T a 's as follows: 

• 0<*(x) = Q(\-i) for an Y x e M d ; 

• r id (x) = Qid(x) for any x e [n] d ; 

• T a jj(u,x,\,y,w) = Q a .ij(u,x + y,\,0,w) for any x,y e [re], u e [re] , v e [n] ; ~' _1 and w e [n] d ~j. 

We leave it to the reader to check that the sequence (T a , Q a )aeS(d) satisfies the formulas (Fj), . . . , (Fg) (and 
hence, is a of-simulation of Q). □ 

Proposition Appendix B.6. For any d>0, ESO(V rf , arity d, half-sorted ) c ESO(V c/ , arity d, sorted ) on coord'' 

Propositions Appendix A. 9 and Appendix B.6 can now be collected in te following result: 
Theorem Appendix B.7. ESO(V c/ , arity d) = ESO(V rf , arity d, sorted ) on coord^ -1 for any d > 1. 



(Fi) min(xd) -> (r,y(x) <-> Q(x )). 

(F 2 ) (-imax(xi) A -wiax(xj)) -» (r a .y(x w ) «-> T a .ij(x (j) )j. 

(F 3 ) min(xi) -> (r«(x) <^ T a jj(xfj. 



{Fa) GhOO « 6(x-). 




(f 6 ) a, l7 (x) « e ff , 7 ( X W). 



(Fi)...(F 6 ) 
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